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operator actions. ACTIN’ s hierarchy is a dynamic 
representation of the operator function model (Figure 3). 


Introduction 

With the emergence of new technology for both human-computer 
interaction and knowledge-based systems, a range of 
opportunities exist to enhance the effectiveness and efficiency of 
controllers of high-risk engineering systems. This paper 
describes the design of an architecture for an operators 
associate-a stand-alone model-based system, designed to 
interact with operators of complex dynamic systems, such as 
airplanes, manned space systems, and satellite ground control 
systems, in ways comparable to that of a human assistant. The 
presentation will have several sections. The first describes the 
OFMspert architecture. The second describes the design and 
empirical validation of OFMspert’s understanding component. 
The third describes the design and validation of OFMspert's 
interactive and control components. The paper concludes with a 
description of current work in which OFMspert provides the 
foundation in the development of an intelligent tutor that 
evolves to an assistant as operator expertise evolves from novice 
to expert. 

OFMspert Architecture 

OFMspert-Operator Function Model (OFM) expert system-is a 
stand-alone knowledge-based system that is intended to 
function as an assistant to a human expert. This philosophy is 
different than many knowledge-based systems in which the 
computer system replaces or operates suggestions. OFMspert is 
intended to be a subordinate to an experienced operator, possibly 
replacing a less skilled assistant. As a result, OFMspert 
includes features such as dynamic allocation of functions 
between the human and computer controllers, interruption of 
OFMspert by the human user, and 'repair of 
misunderstandings. 

OFMspert (Figure 1) has two primary components that enable it 
to understand’ operator activity in the control of a complex 
dynamic system. The first is the operator function model 
(OFM). The OFM is a representation of operator activity in 
dynamic systems that represents the interrelations between 
dynamic system states and operator functions. Each function is 
hierarchically decomposed down to the level of individual 
operator actions. The OFM defines the knowledge base that 
OFMspert uses to hypothesize expectations of operator activities 
and to infer why a given action was undertaken. Figure 2 
depicts a generic OFM 

The second major OFMspert component is a blackboard on 
which OFMspert dynamically constructs expectations of current 
operator function, subfunctions, tasks and actions. The 
blackboard, called ACTIN (actions interpreter), keeps track of 
model-derived expectations and data-derived interpretation of 


ACTIN and the OFM define OFMspert’s understanding 
component. OFMspert’s utility and effectiveness depend on its 
ability to ’understand’ accurately. 

The Validation of OFMspert T s Intent Infcrencing 
(Understanding) Component 

In order to evaluate OFMspert's intent inferencing 
effectiveness two experiments were conducted in the domain of 
satellite ground control. The first experiment compared 
OFMspert interpretations of operator activity with a domain 
expert’s interpretations. The second experiment involved 
verbal protocols in which subjects controlling the system stated 
the reasons for what they were doing; their reasons were then 
compared to OFMspert’s interpretations. In both cases, 
OFMspert’s understanding was quite impressive. Figure 4a 
and 4b summarize the empirical results. Areas of mismatch 
were due primarily to model errors in the OFM (correctable) or 
long-term planning and browsing— operator functions that the 
OFM had not represented. 

We were very pleased with the intent understanding 
component. Based on its understanding capabilities, OFMspert 
was augmented with control properties in order to function as an 
assistant. 

OFMspert as an Assistant 

Based on the OFM and Rasmussen's abstraction hierarchy, a 
user interface to OFMspert was designed. The human operator 
could request a range of assistance from OFMspert. The types of 
assistance were identified based on the operator functions and 
subfunctions defined in the OFM. Each OFMspert function was 
further decomposed into levels of available assistance so that the 
user could dynamically choose how much or how little 
assistance was desired. 

An extensive evaluation of OFMspert as an assistant (Figure 5) 
was conducted, again in the domain of satellite ground control. 
Trained subjects controlled a simulated satellite ground system 
using both OFMspert and a well-trained human assistant. 
Results showed that though the style of use varied, controllers 
with OFMspert as an assistant controlled the system as 
effectively as controllers with a human assistant (Figure 6). 

This experiment provided strong evidence for the possibility of 
using knowledge-based technology to augment operator control 
capabilities. Subject responses indicated that they liked the 
highly interactive and flexible user interface to OFMspert— and, 
in fact, would prefer even more capabilities for dialogue and 
repair of miscommunication. Indeed, for the design of 
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knowledge-based systems for complex domains, the human- 
human metaphor is an intriguing avenue for further research. 

OFMspert as a Tutor that Evolves to an Assistant 

Current research at Georgia Tech examines the use of OFMspert 
as an intelligent tutoring system (ITS) that can evolve to an 
assistant as the user's skills evolve from novice to expert 
(Figure 7). With the OFM, OFMspert provides the domain 
knowledge (static, dynamic, and operational) needed in an 
ITS. In addition, OFMspert’s blackboard, ACTIN, represents 
expected operator activity, interprets actual activity, and is able 
to assess the differences. As such it provides the initial 
definition of the teaching component of an ITS. 

Finally, as a tool that is designed to function both as a teacher 
and as an assistant, OFMspert may be a very viable 
architecture. With two applications, the assistance function 
being long term, it is easier to justify the development costs that 
such systems inevitably incur. From an operations standpoint, 
novice users may be more likely to spend the time interacting 
and using a training system that they know will eventually 
become a tool that they use operationally. 
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ACTIN's Intent Inferencing Structure 
Figure 3 


523 








Figure 5 
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Experiment 1: Average Percentage of Equivalent Interpretations 

Between ACTIN and a Human Domain Expert. 
(Ordered by Rank). 


Experiment 2: Average Percentage Of Equivalent Interpretations 

Between ACTIN And Verbal Reports. 

(Ordered By Rank). 
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Figure 4a 


Figure 4b 
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Comparison of Human Associate and Ally 
Figure 6 
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During Training 



During On- Una Operations 


Architectural evolution of an Intelligent tutor to an operator's assistant 

Figure 7 
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